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Abstract 

We determine the rate region of the Gaussian scalar-help-vector source-coding 
problem under a covariance matrix distortion constraint. The rate region is achieved 
by a Gaussian achievable scheme. We introduce a novel outer bounding technique 
to establish the converse of the main result. Our approach is based on lower bound- 
ing the problem with a potentially reduced dimensional problem by projecting the 
main source and imposing the distortion constraint in certain directions determined 
by the optimal Gaussian scheme. We also prove several properties that the optimal 
solution to the point-to-point rate-distortion problem for a vector Gaussian source 
under a covariance matrix distortion constraint satisfies. These properties play an 
important role in our converse proof. We further establish an outer bound to the 
rate region of the more general problem in which there are distortion constraints 
on both the sources. The outer bound is partially tight in general. We also study its 
tightness in some nontrivial cases. 

Keywords: mutiterminal source coding, one-helper problem, covariance matrix 
distortion constraint, vector Gaussian sources, vector quantization, dimension reduc- 
tion. 

1 Introduction 

We consider the Gaussian scalar-help-vector source-coding problem (sometimes re- 
ferred to as the one-helper problem). The setup of the problem is shown in Fig. 1. The 
first encoder observes a vector Gaussian source that is correlated with a scalar Gaussian 
source observed by the second encoder. The encoders separately send messages about 
their observations to the decoder at rates R\ and i?2, respectively. The decoder uses 
both messages to estimate the first encoder's observations such that a certain distortion 
constraint on the average error covariance matrix of the estimate is satisfied. The goal 
is to determine the rate region of the problem, which is the set of all rate pairs (i?i , R2) 
that allow us to satisfy the distortion constraint. 

Multiterminal source coding have received considerable attention. The study started 
with the work by Slepian and Wolf [ 1 ]. They considered a lossless problem in which 
two correlated memoryless sources must be reproduced by the decoder with arbitrar- 
ily small error probability. It immediately prompted researchers to extend Slepian and 
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Figure 1 : The Gaussian scalar-help-vector source-coding problem 



Wolf's result to lossy problems where the sources must be reconstructed with average 
distortions no more than a fixed amount. Wyner and Ziv [2 J were the first who studied 
the lossy problem where the decoder reconstructs a source within allowable distortion 
with the help of side information about the source. Since then, the problem has been 
extended in several directions by Berger [3], Tung [4|, Berger et al. [5|, Oohama 0, 
Viswanathan and Berger [7], Wagner et al. HQ, Liu and Viswanath [9] and several 
others. 

Oohama [6 1 studied the one-helper problem when both the sources are scalar Gaus- 
sian and gave a complete characterization of the rate region. He used the conditional 
entropy power inequality (EPI), which was also used by Bergmans to determine the 
capacity region of the scalar Gaussian degraded broadcast channel iflOl . to establish 
the converse for the one-helper problem. Oohama's proof suggests a connection be- 
tween these two problems. Recently, Weingarten et al. ifTTI extended Bergmans's 
result to the vector case and determined the capacity region of the vector Gaussian 
multiple-input-multiple-output broadcast channel by introducing the idea of enhance- 
ment. It is natural to expect the enhancement idea to be useful for the vector extension 
of Oohama's result. In this context, Liu and Viswanath [9] studied the one-helper prob- 
lem when both the sources are vector Gaussian and the distortion constraint is on the 
average error covariance matrix. They combined Oohama's converse arguments with 
Weingarten's enhancement idea to obtain a lower bound. However, their lower bound 
is not tight in general because the steps in Oohama's converse proof are in general not 
tight if the sources are vector Gaussian. In particular, the distortion constraint is not 
met with equality in general. In our earlier work lfl2l . we used the enhancement idea 
with an improved outer bounding technique to characterize a portion of boundary of 
the rate region for the vector Gaussian one-helper problem. However, the complete 
characterization of the rate region remains unknown. 

We consider the simplest version of the problem that cannot be solved using ex- 
isting techniques, namely that in which the primary source is a vector and the helper's 
observation is a scalar. For this Gaussian scalar-help-vector source-coding problem, we 
completely determine the rate region using a novel outer bounding technique. We find 
the optimal Gaussian solution and determine the set of directions in which this scheme 
meets the distortion constraint with equality. This set of directions is used to define a 
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Figure 2: A Gaussian achievable scheme 



potentially reduced dimensional problem that lower bounds the original problem. We 
then proceed as Oohama did to obtain a lower bound to the reduced dimensional prob- 
lem. The lower bound thus obtained is achieved by the Gaussian achievable scheme 
depicted in Fig. 2. In this scheme, the first encoder vector quantizes (VQ) its obser- 
vation using a Gaussian test channel as in point-to-point rate-distortion theory. It then 
compresses the quantized values using Slepian-Wolf encoding. The second encoder 
just vector quantizes its observation using another Gaussian test channel. The decoder 
decodes the quantized values and estimates the observations of the first encoder using 
a minimum mean-squared error (MMSE) estimator. 

We then study the more general problem in which there are distortion constraints 
on both the sources. Oohama [6 | and Wagner et al. [8 1 studied the scalar version of the 
problem and their work together characterizes the rate region completely. Following 
Oohama's technique [6|, one can obtain an outer bound to the rate region of the general 
source-coding by considering two one-helper relaxations, each of which can be solved. 
A portion of the resulting outer bound is tight in general, but the rest of it is fairly loose. 
We obtain an improved outer bound and give sufficient conditions for its tightness. 
Our approach is based on splitting the first encoder's rate into two parts. The first 
part of the rate is used to communicate information about the sufficient statistic for Y 
given X, and the second part is used to communicate information about the remaining 
components of X. The first component is analyzed using results for the scalar problem 
with two distortion constraints [6, 8 1. The second part is analyzed using point-to-point 
rate-distortion theory. 

Another contribution of this paper is that we establish several properties that the 
optimal solution to the point-to-point vector Gaussian source coding problem satisfies. 
The core optimization problem here is to maximize the log | • | function over a set of 
positive semidefinite matrices that are no more than two positive definite matrices in a 
positive semidefinite sense. Since this is a convex optimization problem, its optimal so- 
lution must satisfy the necessary and sufficient Karush-Kuhn-Tucker (KKT) conditions 
[13]. By analyzing the KKT conditions, we arrive at several interesting properties that 
the optimal solution satisfies. These properties are used to prove the converse of our 
main result, and they could prove useful elsewhere, even outside network information 
theory. 
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The rest of the paper is organized as follows. Section 2 explains the notations used 
in the paper. In Section 3, we present the mathematical formulation of the problem 
and summarize our main results. Section 4 is devoted to study the core optimization 
problem in the point-to-point rate-distortion theory for vector Gaussian sources under 
covariance matrix distortion constraint. In Section 5, we give the converse proof on 
our main result. Finally in Section 6, we study the generalization of problem in which 
there are distortion constraints on both the sources. 

2 Notations 

We use uppercase to denote random variables and vectors. Boldface is used to dis- 
tinguish vectors from scalars. Arbitrary realizations of random variables and vectors 
are denoted in lowercase. Let {Xj}" =1 be an independent and identically distributed 
(i.i.d.) random process of random vectors. We use X™ to denote {Xj}™ =1 , and x" to 
denote an arbitrary realization {xj}™ =1 . The superscript T denotes matrix transpose. 
We use <j y and o\\ v to denote the variance of Y and the conditional variance of Y 
given V, respectively. The covariance matrix of X is denoted by Kx- The conditional 
covariance matrix of X given Y is denoted by K X |y, an d is defined as 



All vectors are column vectors, and are m-dimensional, unless otherwise stated. We 
use l m to denote an m x m identity matrix. With a little abuse of notation, is used 
to denote both zero vectors and zero matrices of any dimensions. For two positive 
semidefinite matrices A and B, A >p B (A >- B) means that A — B is positive 
semidefinite (definite). Similarly, A B (A < B) means that B A is positive 
semidefinite (definite). All logarithms in this paper are to the base 2. 

3 Problem Formulation and Main Results 

3.1 The Gaussian Scalar-Help-Vector Source- Coding Problem 

Let {(Xj, be a sequence of i.i.d. zero-mean Gaussian random vectors. At each 

time i, Xj and Yi are jointly Gaussian with the covariance matrix Kx and the variance 
try, respectively. Without loss of generality, we can write 



where a is a vector and Nj is a zero-mean Gaussian random vector with the covariance 
matrix Kn- Assume further that Yi is independent of Nj. Therefore, 



The first encoder observes X" and sends a message to the decoder using an encod- 
ing function 



K X |Y = # (X-£(X|Y))(X-£(X|Y)) T . 



X, = aY + N, 



it 



K x = aa T a\ +K N . 



fin) . Rmn ^ j 



1,...,M<">}. 
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Analogously, the second encoder observes Y n and sends a message to the decoder 
using another encoding function 

/ 2 (n) :R n H+{l,...,M 2 (n) }. 
The decoder uses both received messages to estimate X™ using a decoding function 

0< n > : |l, . . . , M} n ^| x {l,...,M 2 (n) } ^R mn . 

Definition 1. A rate-distortion vector (Ri, R 2 , D), vv/zere D z's a positive definite ma- 
trix, is achievable for the Gaussian scalar-help-vector source-coding problem if there 
exists a block length n, encoders f[ n ^ and and a decoder such that 

R t > * logM| n) for all i e {1,2}, and 



1 n r 



n . 



w/zere 



x"=s w (A (n) (x»),/< n) on). 

Lef 7?. Z?e f/ze closure of the set of all achievable rate-distortion vectors. Define 
11(D) = {(i?i,i? 2 ) : (fli.ifc.D) eft}. 

We ca/7 7£(D) f/ze rate region /or the Gaussian scalar-help-vector source-coding prob- 
lem. 

Since we are interested in the covariance matrix distortion constraint, without loss 
of generality we can restrict the decoding function to be the MMSE estimate of X™ 
based on the received messages. Therefore, X™ can be written as 



X" = E 



X"|A (n) (X"),jf } (Y n 



If a = 0, then the problem reduces to the point-to-point vector Gaussian rate-distortion 
problem which can be solved using existing techniques. Therefore, we will assume 
that a 7^ in the rest of the paper. We will assume further that Kx is strictly positive 
definite, since the case when K x is singular can be handled by defining an equivalent 
problem in which the source covariance is strictly positive definite. We can do so by 
applying an invertible transformation. The details of the transformation is presented in 
Appendix A. 

3.2 Rate Region 

Let us define the following set 

1 |aa T cry2 _27? ' 2 + K N | 
n*(D) = {(R 1 ,R 2 ):R 1 > mm -log 1 Y 



k 2 |K| 

s. t. o =<; K =<; D 

K =<; aa T a^2- 2Ii2 +K N }. 

We then have the following theorem. 
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Theorem 1. For every positive definite matrix D 

11(D) =Tl*(D). 

3.3 A Gaussian Achievable Scheme 

There is a natural Gaussian achievable scheme depicted in Fig. 2 that is optimal for 
the problem described in the previous section. Similar schemes have been shown to be 
optimal for other important problems in Gaussian multiterminal source coding EllSl 
Q4] Q3] [3] |5] |7) . We present an overview of the scheme here. The details of the scheme 
can be found in 016). 

Let S(U, V) be the set of zero-mean jointly Gaussian random variables U and V 
such that 

(a) U, X, Y, and V form a Markov chain U -H> X o Y <H> V, and 

(b) K X |ir,v ^ D. 

Consider any (U, V) € S(U, V) and a large block length n. Let R[ = /(X; U) + e, 

where e > 0. To construct the codebook for the first encoder, first generate 2 nRl in- 
dependent codewords U" randomly according to the marginal distribution of U, and 
then uniformly distribute them into 2 nRl bins. The second encoder's codebook is con- 
structed by generating 2 nR2 independent codewords V n randomly according to the 
marginal distribution of V. 

Given a source sequence X™, the first encoder looks for a codeword U n that is 
jointly typical with X", and sends the index b of the bin in which U n belongs. The 
second receiver upon receiving Y n , sends the index of the codeword V n that is jointly 
typical with Y n . The decoder receives the two indices, then looks into the bin b for a 
codeword U" that is jointly typical with V n . The decoder can recover U" and V n with 
high probability as long as 

i?!> J(X;[/|tO 
R2>I(Y;V). 

The decoder then computes the MMSE estimate of the source X™ given the messages 
U n and V n , and (b) above guarantees that this estimate will satisfy the covariance 
matrix distortion constraint. Let 

K G (r>) = {(Ri,R 2 ) ■ there exists (U, V) £ S(U, V) such that 
R 1 >I(K;U\V) 
R 2 >I(Y;V)}. 

The following lemma gives the achievable rate region by using this scheme. 
Lemma 1. The Gaussian achievable scheme achieves 1Zg(D), which satisfies 

K G (D)=K*(T>). 

It immediately follows from the discussion above that the Gaussian achievable 
scheme achieves 1Zg(D). The equality in Lemma 1 is proved later in Section 5 (prob- 
lem Pq and Lemma 3). Theorem 1 and Lemma 1 together imply that 7vLg(D) equals 
the rate region TZ (D). In particular, this proves that the Gaussian achievable scheme 
depicted in Fig. 2 is optimal for this problem. 

The converse proof of Theorem 1 is deferred to Section 5. In the next Section, we 
study an optimization problem which appears in the converse proof of our main result. 



6 



4 The Core Optimization Problem 



In this section, we study the core optimization problem in the point-to-point rate- 
distortion theory for a vector Gaussian source under a covariance matrix distortion 
constraint. In this setup, an i.i.d. zero-mean vector Gaussian source Z™ with a covari- 
ance matrix K z is observed by the encoder which sends a message to the decoder over 
a rate-constrained channel using a function 



j(n) . R mn^ jl, . . . , j . 



The decoder uses the received message to give an estimate Z™ of the source Z" such 
that 



n . 
i=i 



l -H E (Zi-Zi)^-^)' 



where Dz is a positive definite matrix. As explained in Section 3.1, we can assume 
without loss of generality that Kz is strictly positive definite and 



Z n = E 



Z"|/ (n) (Z n ) 



In the single-letter form, the rate-distortion function of the source Z™ is given by 
the optimal value of the following optimization problem 

min I(Z;U) 

u 

subject to K Z |;7 Dz- 

A Gaussian U is the optimal solution to this problem because of the fact that the Gaus- 
sian distribution maximizes the differential entropy for a given covariance matrix. So, 
we just need to optimize the above optimization problem over all Gaussian distribu- 
tions. Equivalently, we have the following matrix optimization problem 

F(D Z ,K Z )= max log|K Z |£/| 

subject to =<; K z \u 4 D z (1) 
Kz|t/ K z . 

Note that we need to impose an additional constraint 

K Z |[7 K z 

because of the fact that the conditional covariance is no more than the unconditional co- 
variance in a positive semidefinite sense. Observe that if any one of the two constraints 
in (1) is inactive, then the other constraint will be met with equality and we will have a 
close form solution to the problem. If both constraints are active, then it is unlikely to 
obtain a close form solution in general. However, we can establish certain properties 
that the optimal solution satisfies. The rest of the section is devoted to establish these 
properties. 

Since the objective of the optimization problem (1) is continuous and 



{(MK Z |c/^Dz and K z{u 4 K z } 
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is a compact set, there exists an optimal solution KJi^ to (1). Furthermore, since (1) 
is a convex optimization problem, we can get the following Lagrangian formulation 

max log jKzi^l + Trace {K Z \ V A - K Z \ U M 1 - K Z |[/M 2 } , 

where A, Mi and M2 are positive semidefinite Lagrange multiplier matrices corre- 
sponding to the constraints K Z |[/ >p 0, K Z |[/ =^ D z and K Z |;y =^ K z , respectively. 
Then KLj, must satisfy the following necessary and sufficient KKT conditions ifPUl 



K*fJ + A* - M\ - M* 2 = 0, (2) 

K* m A* = 0, (3) 

(d z - K* z]u ) Ml = 0, (4) 

(K z - K* z{u ) M* 2 = 0, (5) 

A*,M* l7 M* 2 ^ 0. (6) 



Observe that in the optimization problem (1), the constraint K Z \u ^ is never active, 
so 



A* = 0. (7) 

Since log | • | is strictly convex over the domain of positive definite matrices, KJi^ is 
the unique maximizer of the problem. Let 

q = {(m*,m;)} 

be the set of all pairs (Ml, M 2 ) of Lagrange multiplier matrices that satisfy the KKT 
conditions. Consider any sequence of pairs (M|, M^)^ in Q. By the continuity of the 
KKT conditions, it follows that the limit of this sequence belongs to Q. Therefore, Q 
is a closed set. Equations (2) and (7) together imply that every pair (M^, Mj) in Q is 
such that 

M*+M*=K^, 

which is a fixed matrix. Hence, the set Q is bounded. We thus conclude that Q is a 
compact set. This along with the continuity of the Trace(-) function imply that there 
exists (Mi, M2) in Q that solves the optimization problem 

min Trace (Mi) 
subject to (M;,M;)gQ. (8) 

Since Mi and M 2 are positive semidefinite, we can write their spectral decompositions 

as 

r 

Mi = X ^ S I ( 9 ) 
1 

M 2 =Y,7iUtJ, (10) 

i=i 

where 



8 



(a) < r, I < m, 

(b) Aj, 7j > 0, for all i e {1, . . . , r} and for all j e {1, . . . , I}, and 

(c) {si}[ =1 and {tj}' =1 are sets of orthonormal vectors. 
We have from (4), (5), (9) and (10) that 



r 

(Dz-K* |[7 )^A iSiS f = 0, 
i=i 
i 

(k z -K* z|[/ ) ^7^ = 0, 



?=i 



which imply that 



(d z -K* |(/ ) S! = 0, Vie{l,...,r}, (11) 
(k z - K* z]u ) u = o, Vie (12) 



Define the matrices 



AlSl, V A 2 s 2 , . . . , v ^ s 



S = 

T = [v/Titi, V^t 2 , . . . , VTTt;] . 
Let B be an m x m positive definite matrix. 
Definition 2. A non-zero m x p matrix U ;s B-orthogonal z/ 

U T BU = I p . 

Definition 3. A non-zero m x p matrix U and a non-zero m x q matrix V are cross 
B-orthogonal ;/ 

U T BV = 0. 

We have the following theorem about the optimal solution to the optimization prob- 
lem (1). 

Theorem 2. (a) Ifr > 0, then S T (k z - v J S is strictly positive definite, 

(b) [S, T] is an invertible matrix, 

(c) [S, T] is K^-orthogonal, 

(d) S is Dz -orthogonal, 

(e) T is K z -orthogonal, 

(f) S one/ T are cross Dz -orthogonal, 

(g) S and T are cross Kz-orthogonal. 
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Proof. For part (a), it suffices to show that S T ^K z — S is non-singular. Sup- 

pose otherwise that it is singular. Then there exists ^ e e R r such that 

e T S T (k z -K* |c/ ) Se = 0. 

Let w = Se. We then have 



w 

where a is the i-th component of e, and 



= (13) 
d 

(k z - K* w ) w = 0. (14) 

Let 

A mm = min{Ai,A 2 ,...,A r } (15) 

and pick any e such that 

< e < Tj — jig. (16) 
||w|| 2 

Consider any 0^z£ M m . Let zs be the projection of z on span{S} and z s j_ be the 
projection of z on the space orthogonal to span{S}. We then have 

Z = Z S + Z S J_. 

Now, 

z T MiZ = (z s + z s ±) T Mi (z s + z s j.) 

= ZgMiZs + ZgMiz s i + Zg^MiZs + z^M^gi 
= z|M lZs , (17) 
where (17) follows because 

Miz s ± = 0. 

Similarly, 

w T z = w T z s . (18) 

We now have 

z T (Mi - eww T ) z = z T Miz - e (w T z) 2 

= z|Miz s -e(w T z s ) 2 (19) 
> z|Miz s - e||w|| 2 ||z s j| 2 (20) 



= ^A l (sfz s ) 2 -6||w|| 2 ||zs|| 2 (21) 



=i 



>A mm ^(sfz s ) 2 -6||w|| 2 ||z s || 2 (22) 

i=l 

= A mm ||zs|| 2 -e||w|| 2 ||z s || 2 (23) 
= ||zs|| 2 (A mm - £ ||w|| 2 ) 

> 0, (24) 



where 
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(19) follows from (17) and (18), 

(20) follows from the Cauchy-Schwartz Inequality, 

(21) follows from (9), 

(22) follows from (15), 

(23) follows because 

||z s || 2 = E(^s) 2 , and 

i=l 

(24) follows from (16). 

This proves that — eww T is a positive semidefinite matrix. Let us define the 
matrices 

Mi = Mi - eww T 
M 2 = M 2 + eww T . 

Then 

(i) Mi,M 2 V 0, 

(ii) Mi + M 2 = Mi + M 2 = K*yJ, 
(iii) 

(d z - K* |tr ) Mi = (d z - K* |t/ ) (Mi - eww T ) 

= (d z - K* |£/ ) Mi - (d z - K* z]u ) eww T 

r 

= (d z - K^i^ e e^jSisj (25) 

= 0, (26) 
where (25) and (26) follow from (13) and (11), respectively, and 

(iv) 

(k z - K* |[7 ) M* = (k z - K* |[7 ) (M 2 + 6ww T ) 

= (k z - K z|t/ ) M 2 + (k z - K* eww T 
= 0, (27) 
where (27) follows from (14). 

So, (k. z ^ v , Mi, M 2 ^ satisfies the KKT conditions, and hence is optimal for the opti- 
mization problem (1). But then 

Trace(Mi) = Trace(Mi) — Trace(eww T ) 
< Trace(Mi), 
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which is a contradiction to the assumption that (Mi,M 2 ) solves the optimization 
problem (8). Therefore, S T ( Kz — J S is non-singular. 



The proof of part (b) is similar to that of part (a). We first show by contradiction that 
the columns of [S, T] are linearly independent. Suppose otherwise that they are linearly 
dependent. Note that the columns of S and T are linearly independent. Therefore, there 
exists 

r I 

O^w = ^a,s t = 53 

i=l i=l 

where a/s and 6j's are scalars such that at least one of the a/s and at least one of the 
fej's are nonzero. Pick any e such that 

||w|| 2 

Then as proved in part (a), Mi — eww T is a positive semidefinite matrix and the 
matrices Mi and M 2 defined as before satisfy (i)-(iii) above. Moreover, 

(k z - K z|Lr ) M* = (k z - K* z|[/ ) (M 2 + eww T ) 

= (k z - K* z|[/ ) M 2 + (k z - K* |t/ ) eww T 
i 

= + (k z - K* z{u ) e J2 bibjtitj 

= 0, (28) 



where (28) follows from (12). So, we again have that ^K^^, Mi, M 2 ^ satisfies the 

KKT conditions, and hence as before we have arrived at a contradiction. Therefore, the 
columns of [S, T] are linearly independent which implies that 

r + l<m. (29) 

Next, we have from (2), (7), (9) and (10) that 



K*^ = Mi + M 2 = ^ A iSi sf + ^7it»tf, (30) 

i=l i=l 

which means that 

r + l>m, (31) 
because otherwise K z yJ will be singular. (29) and (31) imply that 

r + I = m, 

which means that [S, T] is a square matrix, and is therefore invertible because it has 
linearly independent columns. 

For part (c), on post-multiplying (30) by K^.ySi, we obtain 

r I 
i=l i=l 
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which can be re-written as 

r I 

si (l - Ax (sj K* |c/ si) ) - ^ A 4 s, (sf K* |[/Sl ) - 2 7i t< (tf K* z|c/Sl ) . (32) 

4 = 2 1=1 

Since the columns of [S, T] are linearly independent by part (b), the coefficients of all 
vectors in (32) must be zero. Therefore, 

Aisf K^ySi = 1, 

sfK* |c/ si=0, Vie{2,...,r}, 
tfK* |c/ si=0, Vie{l,...,/}. 

Likewise, on post-multiplying (30) by K^ u s 2 , . . . , K^Sj. , K^^ti . . . , K^t; and 
then equating all the coefficients to zero, we obtain similar equations. In summary, 





= i, 


v*e{i,.. 


,r}, 




= i, 


v*e{i,.. 


,/}, 




= o, 


v*,je{i, 


■ -,»■},« ^i, 




= 0, 


Vi,je{i, 






= 0, 


Vie{i,.. 


,r},Vje{l, 



which imply that 

[S,T] T K Z|[/ [S,T] =I ro . (33) 

Hence, [S,T] is K^-orthogonal. 

For parts (d) to (g), we have from (11) and (12) that 

D Z S = K^yS, 
K Z T = K Z|C/ T, 

which along with (33) imply 



S T D Z S 


cTt/ * c 

— 3 *^z\u a 




T T K Z T 


= T T K* zlu T 


= Ij 


T T D Z S 


= T T K z|a S 


= o, 


S T K Z T 


- & *^Z\U L 


= 0. 



This completes the proof of Theorem 2. □ 

It is clear from Theorem 2 that span{S} is the set of directions in which the en- 
coder sends information until the distortion constraint is met with equality. Similarly, 
span{T} is the set of directions in which the encoder sends no information and hence 
K z constraint is met with equality in such directions. Note that if S is an empty matrix, 
then the rate-distortion function is zero. 
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5 Converse Proof of the Main Result 



An outline of the converse proof is as follows. We start with a single letter outer bound 
to the rate region 1Z(D). The single letter outer bound defines the main optimization 
problem P that lower bounds the first encoder's achievable rate for fixed D and We 
solve the Gaussian version Pq of the main optimization problem P by restricting the 
solution space to Gaussian distributions. We show that the problem Pq can be reduced 
to a problem similar to (1). Hence, its optimal solution gives two sets of directions S 
and T as discussed in Section 4. The distortion constraint is tight in directions spanned 
by the columns of S. The idea then is to define a potentially reduced dimensional 
problem, namely the reduced main optimization problem P by projecting the main 
source X on S and by imposing the distortion constraint only in directions spanned by 
the columns of S. The reduced main optimization problem P lower bounds the main 
optimization problem P and its optimal solution is Gaussian. Moreover, Pq and P 
have the same optimal values. Therefore, the optimal solution to the main optimization 
problem P is Gaussian. 

Liu and Viswanath gave a single-letter outer bound to the rate region in [9]. We 
arrive at a similar outer bound by using a slightly different outer bounding technique. 

Lemma 2. (Single-letter outer bound) If the rate-distortion vector (i?i,i?2,D) is 
achievable then there exists random variables U and V such that 

Rt > I(X;U\V) 
R2>I{Y;V) 

D >r K X |D,V 

X o Y o V. 

Proof. See Appendix B. □ 
Let us define the main optimization problem P as 

min I(X;U\V) 

u,v 

subject to R 2 > I(Y; V) 

D >r K X |t/,V 

X o Y o V. 

We will show that the optimal solution to P is Gaussian. Let us first restrict the solution 
space to Gaussian distributions. This results in an optimization problem Pq over the 
conditional covariance matrix K-x\u,v an d the conditional variance &y\v Formally, it 
can be defined as 

min 
subject to 



log 



K 



X|V| 



K 



X|E7,V| 

Ri > \ log 

Z Y\V 
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where 



K 



aa T cry| y + K N . 



Let us denote the optimal values of P and Pq by v(P) and v(Pg), respectively. The 
same notation is used to denote the optimal values of other optimization problems 
defined in the paper. We can rewrite Pq as 



min *log|K X |y| - 
subject to i? 2 > ^ log 



1 



(F (D,Kx|v)) 



(34) 



Y\V 

which is a double optimization problem. Note that for a fixed o~ Y ^ v , ^ e i nner opti- 
mization problem turns out to be F (D, K X |v), which was defined in (1). 

Since P G has a continuous objective and a compact feasible set, there exists an opti- 
mal solution (K-x\u*,v* > a Y\v^) t0 ^' wnere U* and V* represent the corresponding 
optimal Gaussian random variables. We now have the following lemma which states 
that it is optimal for the second encoder to use all R 2 bits for sending a message to the 
decoder. 



Lemma 3. There exists an optimal conditional variance <r Y \v* sucn tnat 



>Y\V 



a Y 2' 2R2 



Proof. See Appendix C. 

(34) and Lemma 3 immediately imply that the optimal value of Pq is 



1 



«(P G ) = -log|K X | 



1 



v(F (D,K X | V .)), 



where 



K X |y» = aa T a Y 2- 2R2 



K 



N- 



and K X |(7*.y* is optimal for problem F (D, K X |y») with an optimal value 
v (F (D, K X |y.)) = \ log |K X |^ iV . | . 



(35) 

□ 

(36) 
(37) 

(38) 



As discussed in Section 4, Kx|c/»,y* gives two sets of directions S and T which satisfy 
the properties in Theorem 2. On substituting (38) into (36), we obtain 



v(P G ) 



log 
log 

log 
log 



K 



X|V* I 



K 



x.\u*,v* 



|[S,T] T K X |y4S,T]| 

[S,Tl r Kx|^,v.[S,T]| 

S T K X |y«S S T K x ,y.T 
T T K X |y.S T T Kx|y*T 



S K 



|Im| 

,S 

1/ 



= -log|S J K X |y*S|, 



(39) 

(40) 
(41) 
(42) 
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where 

(39) follows because [S, T] is invertible from Theorem 2(b), 

(40) follows because [S, T] is Kx|c/*,v*-° rm ogonal from Theorem 2(c), and 

(41) follows because T is K X |y* -orthogonal, and S and T are cross K X |y* -orthogonal 
from Theorem 2(e) and 2(g), respectively. 

We now have the following theorem which is central to the converse proof of our main 
result. 

Theorem 3. A Gaussian (U, V) is an optimal solution of the main optimization prob- 
lem P. 

Proof. First note that since restricting the solution space over Gaussian distributions 
can only increase the optimal value of the main optimization problem P, we immedi- 
ately have 

v(P G ) > v(P). 
So, it suffices to prove the reverse inequality 

v(Pg) < v(P). 

Let us define the reduced main optimization problem P as 

min l(S T X;U\V) 

subject to R 2 > I(Y; V) 

S T DS )p S T K X |(7,yS 
S T X <-> Y V. 

We will show that the main optimization problem P is lower bounded by the reduced 
main optimization problem P. Since [S, T] is invertible from Theorem 2(b) and the 
mutual information is non-negative, we have 

I(X-U\V) = l{[S,T] T X;U\v) 

= I(S T X 7 T T X;U\V) 

= I (S T X; U\V) + I (T T X; U\V, S T X) 

> I(S T X;U\V) . (43) 

Note that any (U, V) satisfying 

D V K X |c/,y 
X^Y 



also satisfies 



S T DS > S T Kx|[/,yf 
S T X ^ Y V. 
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Therefore, the feasible set of P is contained in that of P. Moreover, (43) above implies 
that the objective of P is no less than that of P. We therefore have that the reduced 
main optimization problem P lower bounds the main optimization problem P, i.e. 

v(P) > v(P). (44) 

The objective of P can be decomposed as 

/ (S T X; U\V) = / (S T X; U,V) — I (S T X; V) . (45) 

We now define two subproblems that are used to lower bound the reduced main op- 
timization problem P. The first subproblem P(D) minimizes the first mutual infor- 
mation in the right-hand-side of (45) subject to the distortion constraint in P and the 
second subproblem P{R 2 ) maximizes the second mutual information in the right-hand- 
side of (45) subject to the rate constraint and the Markov condition in P. In other words, 
P(D) is defined as 

min I(S T X;U,V) 

u,v v ' 

subject to S T DS ^= S T K X |c/,yS, 

and P(R 2 ) is defined as 

max 7(S T X;y) 

subject to R 2 > I(Y; V) 

S T X Y «■ V. 

It is clear from the decomposition in (45) and from the definitions of P, P(D) and 
P(R 2 ) that P(D) and P(R 2 ) lower bound P, i.e. 

v(P) > v (P(D)) - v (P(i? 2 )) • (46) 

We now give two Lemmas about the optimal solutions to subproblems P(D) and 

Lemma 4. A Gaussian (U,V) with the conditional covariance matrix K X |;/*,v* is 
optimal for the subproblem P(D), and the optimal value is 

«(P(D)) = ilog|S T K x S|. (47) 

Proof. See Appendix D. □ 

Lemma 5. A Gaussian V with the conditional variance o~Y\y* is optimal for the sub- 
problem P(R 2 ), and the optimal value is 

|S T K X S| 



(~, A 1 S K X S 

< p ^) = 2 hs w^M- <48) 

Proof. See Appendix E. □ 
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Substituting (47) and (48) into (46), we get 

v(P) > ^log|S T K X |v-S|. (49) 
From (42), (44) and (49), we have 

v(P) > v(P G ), 

which means that a Gaussian (U, V) is optimal for the main optimization problem 
P. □ 

We are now ready to prove the converse of Theorem 1. Suppose (R\, i?2, D) is 
achievable, then 

Ri > v(P) (50) 

= v(P G ) (51) 

= ^log|K X |^|-^(F(D,K X |^)) (52) 
1 |aa T af,2- 2 ^ +K N | 

=i r 2 log jxj 

s. t. =<; K =<; D (53) 
K =<; aa T cr^2- 2ii2 +K N , 

where 

(50) follows from Lemma 2, 

(51) follows from Theorem 3, 

(52) follows from (36), and 

(53) follows from the definition of F and (37). 

And if (Ri, R2, D) e TZ, then (53) again holds because (52) is continuous in (i? 2 , D). 
This completes the converse proof of Theorem 1. 

6 The General Source-Coding Problem 

In this section, we study the general source-coding problem by imposing separate dis- 
tortion constraints on both the sources. The mathematical formulation of the problem 
remains the same as in Section 3.1. The only change is in the decoder which now 
uses the received messages from the encoders to estimate both X" and Y n using the 
decoding functions 

g[ n) : {l,...,M[ n) } x {l,...,M.< n) } ^R mn , and 

wi B) jx{i ^'j^r, 

respectively. 
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Definition 4. A rate-distortion vector (Ri, R2, D, d), where D is a positive definite 
matrix, is achievable for the general source-coding problem if there exists a block 

length n, encoders /j™' and f^, and a decoder , g^^j such that 
Ri>- logAff for all i € {1,2}, 



n 

n 

i=i 
1 ™ 

d> -Ye 

i=l 



X, — X, I X, — X 



and 



Yi-Yi 



where 



X" = 



X "|/W (x Tl ),/ 2 (Tl) (y™) 



and 



Y' 



92 

E 



(n) An) 



zr (x"),/ 2 (n) on 



Y n \f[ n) (x n ),/ 2 (n) (y n ) 



Lef 7?. fee ?/;e closure of the set of all achievable rate-distortion vectors. Define 
n{T>,d) = {(R U R 2 ) : (Ri,R 2 ,T>,d) e ft} . 

We caZZ 7?. (D, c?) f/ze rafe region for the general source-coding problem. 

We can assume without loss of generality that the components (Xi, . . . , X m ) of 
X and Y are standard normal, and (Xx, Y) is independent of (X 2 , ■ ■ ■ , X m ). Starting 
from any problem, we can get to an equivalent problem with this structure by applying 
an invertible transformation on the sources and by considering the equivalent distortion 
constraints |[T6l[T7l . This can be done as follows. Let 



Ul,U 2 , 



be an orthonormal basis in '. 



where 



Define the matrices 



starting at 
1 



- (^K x 1/2 a 



CTyK- 



-1/2, 



U = [ui,u 2 , . 



T x = U J K 



-1/2 



Then the transformation is given by 



X = T X X 

Y=±Y. 
cry 
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The covariance matrix of X is 



K x = TxKxTi 



U T K~ 1/2 K X K X 1/2 U 



and the cross-covariance between X and Y is 



1 



Kj>y — T x K X y 

CTy 



1 



T v -X/2 



ITK 



x 



= XJ T (cryK x 1/2 aj 

= u T ( P m) 

= (p,0,...,0) T . 
Now, it is easy to verify that the equivalent distortion constraints are 



(4a) 



(7 



n 



;'=i 



X 



Yi-Yi 



X, 



, and 



Since the above transformation is invertible, it does not incur any information loss. We 
therefore have an equivalent structured problem. So from now on, we will assume that 
our original problem has this structure with p being the correlation coefficient between 
X 1 and Y. 



6.1 An Outer Bound 

First note that if there is no distortion constraint between Y n and Y n , then the problem 
reduces to the Gaussian scalar-help-vector source-coding problem, and hence we have 

n(D,d) C K*(D). (54) 

This bound is tight for large i?2 because the distortion constraint between Y n and Y n 
is always satisfied for large i?2- It prompts us to consider another relaxed problem in 
which there is no distortion constraint between X" and X™, and obtain another outer 
bound to the rate region. However, the outer bound thus obtained is not tight in general 
even for large Rj_. This is because the optimal solution to this relaxed problem is such 
that the first encoder sends information about X\ only. The rest of the components 
of X are simply ignored and therefore the distortion constraint between X™ and X" 
is not met in general. We can obtain an improved outer bound to the rate region by 
splitting the first encoder's rate into two. The first split of the rate comes from the rate 
region of the quadratic Gaussian two-encoder source-coding problem for the sources 
X\ and Y under appropriate individual distortion constraints ([6], [8 ]). The second split 
is the point-to-point rate-distortion function for the vector source (X2, ■ ■ ■ , X m ) under 
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an appropriate covariance matrix distortion constraint. By combining this outer bound 
with that in (54), we obtain a composite outer bound to the rate region. 
Let us denote (X 2 , ■ . ■ , X m ) T by X and let 



D 



D 1 h T 
b D 



where Di is a positive number, b is a (m — 1) -dimensional vector and D is a (m — 
1) x (m — 1) positive definite matrix. Let Ri(D) be the point-to-point rate-distortion 
function of the source X under a covariance matrix distortion constraint D. Then from 
the discussion in Section 4, we have 



£i(D) 



-v (F (D, I r 



1 
2 

■^log|D*| 



0) 



(55) 



where D* is the optimal solution to problem F (D, I m _i) defined in (1). Define the 



sets 



K* 2 (d) = | 



(R U R 2 ) : R 2 > ^\og+ 



I f 1 - p 2 + p 2 2 -2(7? 1 -7? 1 (D)) 

d \ 



K: um (D u d) = l(R 1 ,R 2 ):R 1 - i?x(D) + R 2 > - log+ 



where log + x — max(logx, 0), and 

0(Di,d) = l- 



{l-p 2 )P{D u d) 
2B x d 



4p 2 D x d 



We now have the following outer bound to the rate region of the general source-coding 
problem. 

Theorem 4. For every positive definite matrix D and for every positive number d 

n(B,d)cn*(B)nn* 2 (d)nn: um (D 1 ,d). (56) 

Proof. Consider (i?i,i? 2 ) G 1l(D,d). Let d = f[ n) (X n ) and C 2 = f^ } {Y n ). 
Then 

nR 2 > \ogM^ n) 

> H(C 2 ) 
>H(C 2 |Ci) 
>/(F";C 2 |C!) 

= 7(F";C 1 ,C 2 )-7(F";C 1 ) 

> 7(y n ;f") -7(y";Ci), (57) 
nRx > logM 1 (n) 

> H(C\) 
>/(X n ;Ci) 

= 7(X 1 ";C 1 )+7(X";C 1 |X 1 "). (58) 
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We can lower bound the second mutual information in (58) as follows 



I(X. n ;C 1 \X , 1 l ) = I(X n ;C u X?) 

= l(X n ;Ci,X?,Y n ) 
= l{X n ;C u C 2 ,X?,Y* 
>l(X n ;C u C 2 ) 

> /(X";X n ) , 

where (59) follows because 

X" o o Y n . 

Define the following optimization problem 

min -l(x n ;X n ) 



1 x - 

subject to — > E 

n — * 



X,; — X, 



(x,-x) 



(59) 



(60) 



(61) 



This is the point-to-point rate-distortion problem for the source X" under a covariance 
matrix distortion constraint D. Therefore from the discussion in Section 4, a Gaussian 
G\ is optimal for this problem and from (55), the optimal conditional covariance matrix 
is D* and the optimal value Ri(D). From (57), (58), (60) and the definition of the 
optimization problem (61), we obtain that (Ri, R 2 ) satisfies 

R 2 > -I{Y n -Y n ) - ~I(Y n -C x ) 
n n 

1 



fli-iii(D) > ~I{X?;C X ) 



d > 



1 n 

n. £-~t 



Yi-Yi 



By invoking Oohama's lower bounding technique [6 | next, we obtain 



Ri > \ iog H 



i 



l - p 2 + p 2 2- 2 ^~ R ^ 



which implies that 

(R 1 ,R 2 )£TZ* 2 (d). 

We now proceed to lower bound the sum-rate. 

n(R! + R 2 ) > H{C U C 2 ) 

> I(X n ,Y n ;C 1 ,C 2 ) 

= I(X?, Y n ;C 1 ,C 2 )+I(X n ; C u C 2 \X?, Y n ) 
= I(X[\ Y n ; d,C a ) + I(X n ; C x \X?) 

= /(X 1 ",r n ;C 1 ,C 2 )+/(X";X"), 



(62) 



(63) 
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where (63) follows from (60). The sum-rate can be lower bounded further by mini- 
mizing two mutual informations in (63) separately subject to separate distortion con- 
straints. Using the sum-rate lower bounding technique by Wagner et al. [8|, the first 
mutual information is minimized subject to the distortion constraints D\ and d on the 
sources X" and Y", respectively. We omit the details to avoid repetition. Minimizing 
the second mutual information subject to the covariance matrix distortion constraint D 
is the optimization problem (61) again. We therefore conclude that 

R 1 -R 1 (D) + R 2 > ^log + 

which implies that 

(R 1 ,R 2 )eKu m (Di,d). (64) 

(54), (62) and (64) together establish the outer bound (56). This completes the proof of 
Theorem 4. □ 



2B x d 



6.2 Tightness of the Outer Bound 

We will prove that the boundary of the rate region JZ(D,d) partially coincides with 
the boundary of 1Z* (D) in general, coincides with the outer bound (56) completely if 
b = 0, and partially coincides with the boundary of TZ 2 (d) if b ^ and a condition 
holds. Let 

Rl = inf ji* x : Rl > Rl (D) + \ log i- and D - D*^ - _ } . 

We have the following lemma. 

Lemma 6. (a) There exists a positive number R 2 such that 

K{T>,d)r\{R 2 > R* 2 } =K*(D)n{R 2 > R 2 } . (65) 

(b) Ifh = 0, then 

n(D,d) =n*(D)mz* 2 (d)rni* sam (D 1 ,d). m 

(c) Ifh ^ and R{ < oo, then 

1l(D, d) n > Rl} = K 2 {d) n {Rx > R{} . (67) 

Proof. As explained in Section 3.3, the optimal scheme depicted in Fig. 2 is such that 
the second encoder vector quantizes its observation using a Gaussian test channel as 
in point-to-point rate-distortion theory. So, the average distortion between Y n and Y n 
decreases as R 2 increases. Hence, there exists a positive number i?2 such that for any 
R2 > R 2 , the distortion constraint between Y n and Y n is satisfied and therefore the 
region 

n*(T>) n {r 2 > r* 2 } 

is achievable for the general source coding problem. This along with the outer bound 
(54) prove the equality in (65). 
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For part (b), we will first prove that if b = 0, i.e., D is block diagonal, then 



D 1 



11* (T>) = {(R 1 ,R 2 ):R 1 -R 1 (T>)> -log 

The optimization problem in the definition of 1Z* (D) is 

1 , laa^V^- 2 ^ + K N 



(l-p 2 + p 2 2- 2 ^) | 



(68) 



min - log 
K 2 



IKI 



subject to =<; K =<; D 



(69) 



K =<; aa T cry2 _2i?2 + Kj\j. 



Since our problem has a special structure explained above, we have 

aa 7 ^- 27 ^ + K N = Diag{l - p 2 + p 2 2- 2R > , 1, . . . , 1}, 
which is an m x m diagonal matrix. Now, consider any feasible 



K = 



Kx c T 
c K 



where K\ is a positive number, c is a (to — 1) -dimensional non-zero vector and K is a 
(to — 1) x (to — 1) positive definite matrix. Let us define 



K 



K x 
K 



Then, 



|K| = |K|(/Ti -c^K^c) 
< |K|#i 
= |K|. 

Therefore, without loss of optimality, we can restrict the feasible solutions to be of the 
following form 

K x 
K 



K = 



The restricted feasible set 

j(MK=^D and K =<; Diag{l - p 2 + p 2 T 2R * , 1, . . . , 1}| 

is equivalent to 

{0<Xi <min(L>i,l- / 9 2 + / 9 2 2" 2K2 ), 0=^K^D and K^I m _i} 
Now, the objective of the optimization problem (69) can be re-written as 

1 l-p 2 +p 2 2- 2 ^ 1, 1 - p 2 + p 2 2- 2R * 

— lOg = = - lOg T=T 

2 6 |K| 2 8 \VL\Ki 

1, I - p 2 + p 2 2- 2R - 1, 1 

= - log 1 — log . 

2 8 K x 2 K 
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Therefore, the optimal value of problem (69) equals the sum of the optimal values of 
subproblems 



.1, 1 

mm - log 7— 7 
k 2 8 |K| 

subject to =g K ^ D 

K ==; I m _i, 



and 



.1,1 
mm - log — 

k x 2 6 



2o-27?2 



V2 



subject to < K 1 < min (D ll 1 - p 2 + p 2 2~ 2R2 ) . 

The first subproblem is the point-to-point rate-distortion problem for the source X 
under a distortion constraint D, and hence its optimal value is ^(D). The optimal 
value of the second subproblem is 



1 



loe 



1 

D~i 



_ p 2 2 -2R 2 



Thus, the optimal value of the optimization problem (69) is 



A(D) + ^log H 



1 



1-^ + ^2 



2 -2fl 2 \ 



which proves the equality in (68). It is now easy to verify that the outer bound (56) 
coincides with the shifted boundary of the rate region of the quadratic Gaussian two- 
encoder source-coding problem [8|, where the shift is by the amount R~i(D) in the 
direction of R\ axis. So, by using the point-to-point rate-distortion optimal code for 
the source X in conjunction with the separation-based optimal scheme for the sources 
X\ and Y (8), we can achieve the outer bound. We therefore have the equality in (66). 
For part (c), it suffices to show that if the conditions in Lemma 6(c) hold, then 



2-2(^-^(6)) 
D* 



D 1 
b 



D 



D. 



(70) 



This will imply that the region 



K* 2 (d) n {R t > Rl} 



is achievable for the general source-coding problem by using a scheme in which the 
source X is encoded and decoded as in the point-to-point rate-distortion theory, and 
the sources X\ and Y are encoded and decoded as in the scalar Gaussian one-helper 
problem |6|, treating Y as the main source and X\ as the helper. Consider any 
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where y is a scalar and z is a (m — 1) -dimensional vector. Then 




) 



y 2 (d x - 2- 2 («i- a *CB))) + 2y(z T b) + z t ( fi _ D*) z 




(71) 



> 0, 



(72) 



where (71) and (72) follow from the conditions in Lemma 6(c). This implies that (70) 
holds, and hence we have the equality in (67). This completes the proof of Lemma 



7 Conclusion 

We determined the rate region of the Gaussian scalar-help-vector source-coding prob- 
lem, and proved that the Gaussian achievable scheme is optimal for the problem. We 
introduced a novel way of establishing the converse. Our approach involved lower 
bounding the problem with a potentially reduced dimensional problem by projecting 
the main source and imposing the distortion constraints in certain directions determined 
by the optimal Gaussian scheme. The core optimization problem for the converse turns 
out to be the point-to-point rate-distortion problem for a vector Gaussian source under 
a covariance matrix distortion constraint. The properties satisfied by the optimal solu- 
tion to the point-to-point problem plays an important role in our converse arguments. 
We also generalized our work to the general source coding problem in which there 
are distortion constraints on both the sources, and obtained an outer bound to the rate 
region. The outer bound is partially tight in general. We also studied its tightness in 
some nontrivial cases. 

Appendix A 

We will use a transformation similar to the one used by Liu and Viswanath |9j . Let 
us suppose that the rank of Kx is p < m. The spectral decomposition of Kx is 



6. 



□ 



K x = QSQ T , 



where X is an orthogonal matrix and 



£ = Diag(ai, . . . , a p , 0, . . . , 0) 
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is a diagonal matrix. Let 



Q T DQ 



A B T 
B C 



where A, B and C are submatrices of dimensions p x p, (m — p) x p and (to — p) x 
(to — p), respectively and let 



Tp-1 



T x = 

Then the transformation is give by 

X = 



Ip B 1 C 



Q 



x 
x 



T X X, 



where X is a p-dimensional random vector. The covariance matrix of X is 



K x — T X K X T X 



'-p 




B T C 1 

Im— p 
Tp-1 



I p B C 



Q K X Q 
I 



Ip 

-C _1 B I„, 



p 

-C^B I 





711— p 



= S, 



which means that K x = 0, and hence X is deterministic. Now, 



T X DT X 



I D B C 



Q T DQ 



C^B I 





m—p 



I p B T C 1 

^-m—p 

A — B T C *B 
c 



A B T 
B C 



-C- ] B I 







Since Tx is invertible, the distortion constraint is equivalent to 

> 'e fx, - xj;X:--X;' : 

n ■ 



1 n r 

TxDT x^"E £ (Xi-Xi) (Xi-Xj)' 

i=l L 



1 " 

n ^ 



Xi — Xj 




Xi — Xi 




1 v^n TP 



^Xj — Xj^ ^Xi — x^ 









(73) 



(74) 



Since A — B T C X B and C are strictly positive definite, (73) and (74) imply that the 
equivalent distortion constraint is 

i n r 

A - B T C *B >~Y. E ( X * - X «) ( X * - X 
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Finally, since X™ is deterministic, the first encoder can send information about X™ 
only. We therefore have an equivalent Gaussian scalar-help-vector source-coding prob- 
lem in which the covariance matrix of the first encoder's observations 

K x = Diag(ai, . . . ,a p ) 

is positive definite. Hence, we can assume without loss of generality that Kx is strictly 
positive definite. 

Appendix B: Proof of Lemma 2 

Assume (R U R 2 ,T>) is achievable. Let d = f[ n) (X™) and C 2 = f^ n) (Y n ). We 
now have 

nR 2 > logM 2 (n) 
> H(C 2 ) 
>I(Y n ;C 2 ) 

n 

= Y / [HY l \Y t - 1 )-h(Y l \Y*-\C 2 )] 
»=i 

n 

= Wi) - h(Y^-\Y l -\C 2 )] (75) 

i=l 
n 

>Y J W l )-h(Y l \lC-\C 2 )\ (76) 
»=i 

n 

= J2[KY i )-h(Y i \V i )} (77) 

i=l 
n 

= Y,I{Yi\Vi), (78) 
»=i 

where 

(75) follows from the Markov condition Y t C 2 ) «• X*" 1 , 

(76) follows because conditioning reduces differential entropy, and 

(77) follows by letting V t = (X*" 1 , C 2 ). 
Furthermore 

ni2i > logM 1 (n) 

> H(C\) 
>H(C\\C 2 ) 

> J(X";Ci|C 2 ) 

n 

= ^/(X 4 ;C 1 |X J - 1 ,C 2 ) 

i=l 
n 

= 2/(X j ;Ci|Vi). (79) 
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Let Q be a time-sharing random variable uniformly distributed over {1, . . . , n} and 
independent of all other random variables and random vectors. Let us define 

U = (Q,C 1 ) 

V = (Q,V Q ) 

Y = Y Q 

x = x Q . 

Then we have from (78) that 

n 

r 2 > - Y^m-M) 

n ^ 

i=l 

= I(Y Q :V Q \Q) 

= I(Y Q ;V Q ,Q)-I(Y Q ;Q) 

= I{Y Q] V Q ,Q) 

= I(Y;V) 

and from (79) that 

1 ™ 

Ri > - y)/(x i; ci|vi) 

= 7(Xq;C 1 |F q ,Q) 
= I(X Q ;C\,Q\V Q ,Q) 
= I(X;U\V). 
Now for each i e {1, . . . , n}, we have a Markov chain 

x, = by, + Nj y; ^ Vi = (x*- 1 , c 2 ) 

because Nj is independent of (X' _1 ,C 2 ). Thus, X, Y and V also form a Markov 
chain 

x h y h v. 

Finally, we have 

i=l 
1 " 

= -r£[(Xj - ^(X i |C 1 ,C 2 ))(X i - ^(XilCi, C 2 )) T ] 
1 ™ 

> - £ £ [(X, - £(Xi|Ci, C 2 , X'" 1 ))(X i - £7(Xi|Ci, C 2 , X^ 1 )) 7 ] (80) 
1 " 

= -V£[(Xi - ^(XilCi, Vflpb - E(X i \C 1 ,V i )) T ] 
i=i 

= E [(X Q - £7(Xq|Ci, Vq))(Xq - ^XqICi.Vo)) 3 ] 

= ^ [(Xq - £'(Xq|Ci, Vq, Q))(Xq - E(Xq\C\, Vq, Q)) t ] 

= E[(X- E(X\U, V))(X - E(X\U, V)) T ] 
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where (80) follows because conditioning reduces the covariance of the error in a posi- 
tive semidefinite sense. 

Appendix C: Proof of Lemma 3 

First note that if the second encoder's message alone is sufficient to meet the distortion 
constraint, then R\ = 0. Hence, the second encoder can use all available rate i?2 for the 
transmission of a message to the decoder and therefore (Jy\v = °y2~ 2fl2 is optimal 
in this case. Consider now the other case when both encoders need to send messages 
to the decoder. We will first show that if we restrict the solution space of P to feasible 
jointly Gaussian distributions, then we can assume without loss of generality that we 
have the long Markov chain 

U <-> X <H- Y -H- V. 

It suffices to show the same for Gaussian U and V such that (Tyiy is feasible and 

K X |y y is the corresponding optimal solution to problem F (D,K x iy) defined in 

(1). Then from (34), (&y^y, K X |jj ; y^ is a candidate to be an optimal solution to 

Pg- From Section 4, K X |£/ ; y gives two sets of directions S and T which satisfy the 
properties in Theorem 2. Let us define 

?7 = S T X + W, 

where W is a zero-mean Gaussian random vector independent of X and has a covari- 
ance matrix Kw that satisfies 

(S T K X| ^S) _1 = (S T K x| yS) _1 + K w *. (81) 
Note that Kw is strictly positive definite because 

S T K X |[/ V S -< S T K X |yS 

from Theorem 2(a). The conditional covariance of X given (U, V) can be expressed 
as 

K*\u,v = Kxiv- - E (XU T \V) KZ^E (uX T \v) 

— K x|y - K x|y s ( sTK x|y s + K w) S T K X |y, 

which implies 

S T K X |jj yS = S T K X |yS — S T K X |yS (S T K X |yS + Kw) S T K X |yS 

= [(s^ysy 1 +K^y 1 

= S T K X |pyS, (82) 
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where (82) follows from (81), and 

tTk x|!/,v t = TTK x|y T - tTk x|v- s ( sTk x|v s + K w) S T K X |yT 

= T T K X| ^T (83) 

= I; (84) 

= T T K x|c7i v-T, (85) 

tTk x|(7,v-^ = tTk x|v-S - T T K X |^S (S T K X |yS + K w ) S T K X |yS 

= (86) 
= T T K X | P ^S, (87) 

where 

(83) and (86) follow because S and T are cross K X |y -orthogonal from Theorem 

2(g), 

(84) follows because T is K X |y-orthogonal from Theorem 2(e), 

(85) follows because T is K X |p y-orthogonal from Theorem 2(c), and 

(87) follows because S and T are cross K X |p y-orthogonal from Theorem 2(c). 
In summary, we have 

[S,Tf K X| ^[S,T] = [S,TfK x|[/! v-[S,T], 

and hence 

K X|t>,V = K X|£/,V; 

because [S, T] is invertible from Theorem 2(b). This proves that we can assume with- 
out loss of generality that U is of the following form 

[7 = S T X + W. 

and therefore we have the following long Markov chain 

U^X^Y <-> V. 

Hence, without loss of generality, we can assume that any feasible solution to Pq, 
in particular (U* , V*) satisfies the long Markov chain. Next, because of the second 
encoder's rate constraint, we have 

<Ty\V* > °Y 2 ~ 2R2 - (88) 

We want to prove equality in (88). Suppose otherwise that 

>a 2 Y 2- 2R \ (89) 

Then there exists a zero-mean Gaussian random variable V such that for some e > 0, 
the conditional variance of Y given V 



a Y\V ~ Y\V* — e °Y Z ' 
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and U*, X, Y, V and V* form a Markov chain 



(90) 



We therefore have 



K 



X|(7*,V 



K 



x|i/*,v,v* 
x|[/*,v* 



and 



which means that (k.^ v , v ,a 
following chain of inequalities 

I(U*;X\V)=I(U 
= I(U 
< I(U 

= I(U 
= I(U 



aa a y ^y + K N , 



Y \v) * s f easl ble for problem Pq. We now have the 



X\V,V*) 

X,V\V*)-I(U*;V\V*) 
X,V\V*) 

X\V*)+I(U*;V\V*,X) 
X|V*), 



(91) 
(92) 
(93) 



where 

(91) and (93) follows from the Markov condition in (90), and 

(92) follows because 



I(U*;V\V*) 



1 K^.| V . 
log ■ 



2 



\ log 



|Kt/*|v* | 



|S T K X |y.S + K w | 



S T (aa T 4 |y , +K N ) S + K w 



S T (aa r 4^ + K N ) S + K w 



> 0. 



We have arrived at a contradiction to the assumption that U* and V* are the optimal 
Gaussian random variables. Therefore, the supposition (89) is wrong, and hence (88) 
holds with equality. 



Appendix D: Proof of Lemma 4 
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We have 

h (S T X\U, V) < \ log ((27re) r | S r K X | ^ ^ S | ) (94) 
< ilog((27r e ) r |S T DS|), (95) 

where 

(94) follows from the fact the Gaussian distribution maximizes the differential en- 
tropy for a given covariance matrix, and 

(95) follows from the distortion constraint. 

The inequalities (94) and (95) are tight if X, U, and V are jointly with the conditional 
covariance matrix Kxij/.v sucn tnat 

S T K X |c/,yS = S T DS. (96) 

Since Kx|t/*.v* satisfies (96), we conclude that a Gaussian (U, V) with the conditional 
covariance matrix K X |(y*,y* is optimal for subproblem P(D), and the optimal value 
is 

v (P(D)) = h (S T X) - ^ log ((27re) r |S T DS|) 

= ilog((2^e) r |S T K X S|) - ilog((27re) r |Ir|) (97) 
= ilog|S T K x S|, 
where (97) follows because S is D-orthogonal from Theorem 2(d). 

Appendix E: Proof of Lemma 5 

First note that if 

S T a = 0, 

then 

S T X = S T (aY + N) = S T N, 

which means that 

v{P(R 2 )) = 0, 

because Y is independent of N, and we have a Markov condition S T X <-> Y -H- V. 
So, any V including a Gaussian one with the conditional variance cr^y, is optimal for 

subproblem P(R 2 ). Therefore, Lemma 5 is trivially true in this case. Let us assume 
now that 

S T a ^ 0, 
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and let 

Ui,U 2) ... ,u r 
be an orthonormal basis in M. r starting at 

u 1 = i(S T K x S)- 1/2 S T a, 

c v ' 



where 



Define the matrices 



and the transformation 



(S T K x S)~ 1/2 S T a 
U = [ui,u 2 , . . . ,u r ] 



T x = U T (S T K X S) 



-1/2 



X = T x (S T X) . 
Then the covariance matrix of X is 

K x = T x (S T K X S) T£ 

= U T (S T K X S)" 1/2 (S T K X S) (S T K X S)" 1/2 U 
= U T U 
I r , 

and the cross-covariance matrix between X and Y is 
K xy = TxS T Kxy 



= U T (S T K x S)" 1/2 S T a ( 7f. 



U T Ul 



(S T K X S)" 1/2 S T a 



4 



= (c4,o,...,o) . 

This means that under this transformation, X has i.i.d standard normal components, 
and Y is correlated with Xi only and is uncorrected with the rest of the components 
of X. Since the transformation matrix T x is full rank, we have 

7(S T X;y) =I(X;V) 



= I(X 1 ;V)+I(X 2 ,...,X r ;V\X 1 ) 
= I(X 1 ;V), 



(98) 



where (98) follows because {X 2l ■ . . , X r ) is independent of (V, X\). It is also clear 
that Xi, Y and V form a Markov chain 

Xi Y V. 

Therefore, the subproblem P(i? 2 ) is equivalent to the following problem 

max I(X i; V) 

v 

subject to R 2 > I(Y; V) 
X X ^Y <*V, 
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which is a well-known problem in scalar Gaussian lossy one-helper problem. Oohama 
in H showed that a Gaussian V with the conditional variance o Y \v* is optimal for 
this problem. Hence, the same Gaussian solution is optimal for P(R*i) too. Thus, the 
optimal solution to P{R%) is Gaussian with the conditional variance cr Y ^ v , and the 
optimal value is 



v (P(ifc)) 



- log 
2 S 





S T K X S 










s 



1, |S T K X S 

= o lo g 



S r K X | V .S|' 



(99) 



where (99) follows from (37). 
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